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This paper describes a LISP function, ERASER, to be used in the 
process of recognizing objects by computer . It is a pie-processor 
to a program called SEE which finds whole bodies in scenes* A 
short description of SEE and the required data-form for a scene 
is given. SEE is simulated for five different scenes to demonstrate 
the effects of shadows on its operation* The function ERASER is 
explained through its sequence of operation, the heuristics used 
and detailed results for test cases. Finally, a "how to use it" 
section describes the data required to be on the property lists of 
the vertices in the scene, and the cruft that ERASER puts on these 
p-lists as it operates. 
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Introduction 

■ 

This paper describes a LISP function, ERASER, used as an aid 
in the interpreting of a real-life scene- The scenes considered 
are randctr groupings of rectangular blocks of various sizes. A 
representation of such a scene, useful for manipulation, is in the 
form of a network of links, showing how, when viewed from some angle, 
the vertices of the solid objects are connected- The network is a 
mapping of the three-dimensional scene into a two-dimensional repre- 
sentation and would be derived previously, by an exterior program, 
from vidisector data. 

ERASER is used to remove vertices and links caused by shadows, 
from the network. With the superfluous data removed, the network 
can be analyzed more successfully by SEE, a function that finds 
bodies in scenes* 

Background Information 

To allow a proper perspective for understanding what Q*ASER 
does and how it operates, a short description of another program, 
which handles the same data-type, is necessary. For a more detailed 
discussion, see references fl) and (2) . 

The present process used for the recognition of objects by 
vision encompasses three steps i 
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1. A program to convert the light intensity information of a 
visual scene, obtained from a vidisector, into a network of vertices. 
In a LISP representation, the network is a list, whose elements pro- 
vide information about each vertex visible to the vidisector. This 
information would be: the name of the vertex, the x- and y-coordi- 
nates, and the neighboring vertices (i.e. those vertices in tho net- 
work connected to the named vertex). For example, a cube might be: 

( A (1 1) (F B) 

B (1 6) (A E C) 

C (4 8) (B D) 

D (9 8) (G C E) 

E (6 6) (F D B) 

F (6 1) (A G E) 

G (9 3) (F D) ) 

Figure la is a more natural, equivalent way of representing this 
same cube. 

2. A program for processing ths input data and determining 

whole bodies. 

i 

3. A matching program to classify the bodies as we 11 -under stood 
objects — for which the system has convenient and easy to manipulate 
models . 

The second step is accctnplished by SEE, a Ph.D. thesis by 
Adolfo Guzman, in conjunction with a pre-processor . The pre-processor 
operates on the vertex network and derives information necessary for 
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SEE. Regions are found and labelled* Bach vertex is classified as 
being one of eight types (see Figure 2) — - L, T, X, K, ARROW, FORK, 
MUIiTI, or PEAK -- and information characteristic to that type is sup* 
plied. An L is a vertex linked to two other vertices. A vertex 
linked to three vertices can be a T, a FORK, or an ARROW. The vertex 
with four neighboring vertices is either an x.or a K- A vertex not 
fitting one of the above categories is a MULTI, or if the y- coordinate 
is greater than that of any of the linking vertices the special case 
PEAK is used. For Figure 1: 

Regions: 1, 2 t 3 

« 

Vertices of type L: A, C, G 
Vertices of type FORK: E 
Vertices of type ARROW: B, D, F 

Operating on the network and the information supplied by the pre- 
processor, SEE makes reasonable guesses concerning which regions 
form bodies. The actual process is to create a network of linked 
regions frcxa the network of linked vertices. The new links, between 
the regions of the scene, are contructed from a systematic analysis 
of the vertices of the scene. Two degrees of association between the 
regions are possible. These are designated "strong" and "weak" links. 
Three functions in SEE merge regions to form nucleia according to 
the links that have been formed* GLOBAL requires that regions be con- 
nected by at least two strong links to be joined as a nucleus. LOCAL 
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joins regions having one strong link supported by a weak link. A 
single region having only one strong link, which connects it to only 
one nucleus, will be connected with that nucleus by SINGLEBODY. When 
all possible nucleia have been formed, any remaining links are ignored 
and SEE outputs each nucleus as being a body — listing the regions 
comprising the bodies. 

Except for recently explored and documented weaknesses or special 
problems (see reference (3)), SEE performs well in noise-free situa- 
tions. The motivation for ERASER is to enable SEE to handle more 
complicated and realistic situations, namely those having shadows. 

Allowing shadows, the heuristics used by SEE cause improper pro* 
cessing of the scene. This is understandable since these heuristics 
are local; specifically, observed characteristics of the transforma- 
tions resulting from the mapping of three-space into two-space. A 
FORK is a front-on view of a corner; an ARROW is a side view of a 
corner. The T-type vertex can be either a degenerate view of an edge 
and corner or two regions of one body obscured by another distinct 
body, depending upon: the location of the background relative to the 
T, the existence of an "opposite" and "facing" T vertex (such a pair 
of matching Ts are shown in Figure lb), or the slopes of segments 
bounding the regions that the T bounds. An X occurs as the interior 
edge of stacked blocks. In addition, some special case heuristics 
are used which help prevent links that would be formed due to optical 
illusions or point— of— view problems. 
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Although a heuristic may Involve a search for a vertex quite 
removed foi the vertex being analyzed, aa in the case of the matching 
Ts, no explicit model or conjecture ie made concerning what type of 
body ie being pieced together- SEE'e gueafieo about the structure 
of material bodies is baaed on from one to three edges of a single 
region* 

Kith SEE in its present form, there are two obvious types of 
action to be expected for scenes having shadows: 

1* In conservative situations, with insufficient links between 
real body regions and shadow regions, the shadow region is simply 
ignored and becomes a one-region body. 

2. When links between shadow and body exist, the shadow is 
absorbed into the body and thereby complicates the body. It nakes 
the job of the pattern-matching program more difficult. 

Simulation of SEE 

SEE was simulated for five sample scenes of varying complexity, 
and the above-mentioned actions are observed to have occurred- The 
scenes are shown in Figures 3, 5, 7, 9, and 11. A diagram showing 
three successive stages in the operation of SEE follow the scenes 
(Figures 4, 6, 8, 10, and 12). Each such diagram shows all the links 
formed between the regions of the scene, the nucleia formed by the 
merging process, and the results as they might be printed out by SEE. 




Figure 3 . 
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Figure 7 . 
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Figure 10. 
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The notation for these diagrams is as follows: 

the numbers correspond to regions 

solid lines are strong links 

dotted lines are weak links 

the letters are the vertices causing the link. 
The scenes considered in no way caused errors due to degener- 
ate viewing conditions or optical illusions. The five cases show 
how the heuristics for solid objects are just unable to cope with 
shadows- The most devastating action caused by the presence of 
shadows is to divide a body into several pieces which would other- 
wise have been correctly recognized as continuous. This division 
is caused primarily by the heuristics connected with the T and X 
vertixee (overlapping bodies and interior edges) — very basic 
heuristics which are quite powerful in noise-free eituations- 

Except for the difficulties created for later processing in 
the vision recognition sequence, the action of connecting a shadow 
with its body is technically correct. I think an indication of 
how three-dimensionally oriented SEE's heuristics are is the fact 
that shadows are merged together into nucleia as GEE tries to pull 
the two-dimensional shadows into a real body. Again, this is 
caused by the X and T heuristics. 

The approach used in the design of ERASER was to categorize 
the characteristics of shadows into a set of heuristics that could 
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be applied to the vertex network prior to SEE. In this way the 
features caused by shadows, which run counter to the heuristics 
embodied in SEE, are completely removed from the network. 



General features of ERASER 

ERASER is to be used just prior to SEE, after the pre-processor 
has examined the simple vertex network and established the neces- 
sary vertex information* ERASER is handed a list of all the dark 
regions, the thought being that any shadows present in the scene 
will be contained in this list. Such information about the regions 
should be readily available from the light gradients given by the 
vidisector. Not all the dark regions are shadows, however, since 
the face of objects turned away from the light are as dark as 
shadows* And, some of the line segments bounding a shadow region 
also bound a real body region. Nevertheless, the heuristics in 
ERASER do not remove the real body lines, in most cases. Therefore, 
it would be reasonable to use ERASER as a predicate to decide if a 
given dark region were a shadow or not. 

ERASER does not need to know global information about the 
scene, for example which body is causing the shadow, or light gradi- 
ents across boundaries of the regions. The heuristics are entirely 
local and involve two, or at most three, connected vertices. Two 
types of heuristics are found in ERASER, grouped as Series One and 
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Series ivo. Series One is further divided into primary and secon- 
dary. 

The biases toward data that are built into ERASER are similar 
to thoso in SEE- With currently no mechanism set up to handle 
curved lines , ERASER assumes the connections in the vertex network 
are straight lines. Also, several of the predicate tests are 
designed with a right shadow projection in mind. However, a set 
for left-projected shadows could be made with a choice of which to 
use based on additional information supplied to ERASER, The pro- 
jection-dependent tests are only used to effect a smoother operation 
and are not used in the decision to remove a suspected shadow. 

Thu operational procedure is straightforward. ERASER examines 
the dark regions in the scene, one by one. Tracing counterclockwise 
around the perimeter of each such region, the sequence of vertex 
types encountered is analyzed- The primary heuristics of Series 
One determine if a line segment of the perimeter, connecting two 
vertices, is a shadow line or a part of a body. If believed to be 
a shadow, the segment is marked for later removal from the network. 
When all the vertices of a region have been considered in this 
manner a new region is analyzed* After the last region has been 
examined a function is called which removes the marked segments 
from the vertex network. As shown in Figure 13, vertices change 
type (J goes from an "I" to a straight "L H , F goes from "MULTI* 
to "ARROW") or disappear (H, I — shadow vertices of typo "L"). 
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Figure 13 . 
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vertex network can now be collected and submitted to the pre- 
processor for reclassification of the edified vertices. W. 
calling BEE. For Figure 13j 

Before ERASER <F (X Y) <H G E A) > 

(G (X Y) (F J)) 

4 

(H {X Y) (F I)) 
(X (X Y) (H J)) 
( J (X Y) (G I D) > 

After ERASER (F <x y) <G EA>) 

(G (x y) (F J)) 
<H (x y) HXL) 

(i (x y) NIL' 

(j (x y> <G D)> 



The secondary heuristics of Series One give an -mediate 
valuation of the priory huristics. «« the primary heuristics 
are finished and before the next vertex of the perimeter is brought 
into consideration. ERASER is looKing at two line segments (see 
F igure 12.1a,. It this point the secondary heuristics are called 
ta to reconsider the decision made regarding the previous 11- •*>- 
„ent. This decision is either accepted or reversed, based upon 
the current line segment (two vertices, Just examined and the deci- 
sion regarding it. This set of rules is most useful for removing 
the leading edge of a shadow (segment (r » in Figure 13, or for 
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replacing part of a T that has been too quickly removed (segment 
(AN ak) in Figure 11) - Modification of the vertex network tends 
to be art irreversible procedure and it is due to the secondary 
heuristics that the segments are only marked* and not removed, 
until all the regions of a scene have been examined* 

To effect the operation of the Series One heuristics, a set 
of pattern-matching predicates were derived, based on two facts 
about shadows that allow their quick and easy recognition: 

1. the abundance of La, Ts, and Xs bounding shadow regions. 

2. the pattern of the occurrence of these three types of 
vertices in conjunction with the occurrence of the other possible 
types of vertices bounding any particular region. 

These two facts are relevant in the following ways. First, 
the three vertex types mentioned are those caused most often by 
shadows. The L is the projection of a corner of a block located 
above the plane of the L- Normal intersections of shadows with 
bodies appear as Ts or Xs, e.g. when a body blocks the view, or 
a shadow or one body falls on two faces of another body. Second, 
the typical shadow has the following sequence of vertices as part 
of its perimeter (see Figure 13); ARROW - L - L - T. Of course, 
the viewing angle can, and does, affect the actual vertices of the 
shadow, with the exception of the Ls and Xs (other than obscuring 
them). Regular deviations from the typical case are to: replace 
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the T with an X or a K; insert a series of Ts and/or Xs between 
the Ls? or replace the ARROW with practically any type of vertex. 
When considering two vertices, one of which is of the type T, K, 
or X, it is necessary to know if the other vertex is in the con- 
tinuing line of this T, K, or X. For example, in Figure 2, ver- 
tices A and C are in the continuing line of the "T" vertex B; ver- 
tices D and E are in the continuing line of "X" vertex C- A line 
segment that forms part of this continuing line cannot be a shadow 
line and is therefore never removed. Or, if removed by Series 
One/primary, it will be replaced by Series One/secondary. This 
rule is the basis for determining shadows and is specifically what 
the predicate tests for. 

The Series Two heuristics are an attempt to make the analysis 
of the dark regions more realistic, i.e. more like what I imagine 
human analysis to be. Taking a hint frcm the way the X heuristic 
of SEE tends to pull shadows together, ERASER, through Series Two, 
tries to piece together dark regions to form a continuous shadow. 
As a result of these two types of heuristics the operation of 
ERASER is quite realistic. 

An example: ERASER follows the shadow of the upright body 

in Figure 9 from the supporting surface, up the face of the solid 
object, across the top of the object, and then looks for the con- 
tinuation of the shadow on the supporting surface, behind the long 
body. Being able to piece together the dark regions in this way 
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seems to yield a potentially valuable piece of information abut the 
scene itself. One additional device could be added to improve this 
general semantic processing* ERASER should recognize that vertex All 
presents an unnatural change of direction and that such changes of 
direction can be attributed to intersecting shadows, it would then 
jump up to vertex AV to continue its analysis, returning later to All 
as the starting point for the second shadow. To be able to do this 
efficiently would require either more global heuristics or some type 
of pre-ERASER processing of the scene. 



Heuristics of ERASER 

The primary and secondary heuristics embodied in the Series One 
group can be summarized as follows: (notation — CV ■ "nm", the cur- 
rent vertex is type "nm"; PV - M nra", the previous vertex is type "nm") 

Primary 
1. CV = L 

a. PV - FORK, ARROW, PEAK, K or MULTIt no action 

b. PV * L; remove the line 

c. PV = T and CV is not in the continuing line of the T; remove 

t 

the line 
2- CV - T 

a. PV = X or T and neither vertex is in the continuing line of 
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of the other; remove the line 

b. PV = L and PV is not in the continuing line of CV; remove 
the line 

c, PV ■ FORK, ARROW, PEAK, K or MULTI; no action 

3. CV - X 

a. PV ■ any and PV is not in the continuing line of CVr remove 
the line 

4. CV * K is treated as a special case of CV ■ T above 

5. CV = PORK, ARROW, PEAK or MULTI j no action 

Secondary 

6. If PV = T and the previous line was removed and the current 

vertex is in the continuing line o£ the PVj replace the previous 
line 

7. If PV = L and the previous line was not removed and the current 

line has been removed; remove the previous line 
8- If PV - X and CV is not in the continuing line of the PV and the 
current line has not already been removed: remove the current 
line. 

The two heuristics of Series Two can be stated as follows: 
1. When ERASER discovers the current vertex to be an X (Figure 
12.1b, vertex A) , rather than turn left as it does for, say, a MULTI 
(Figure 12.1b, vertex C) it continues on across the continuing line 
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of the X and adds the perimeter of the region left of segment AB to 
the perime.er of the region currently being analysed. 

2- A T vertex (Figure 12.1c, vertex A) causes ERASER to 
look right along the continuing line of vertex A for a T whose up* 
right is oriented opposite to that of the current vertex (as shown 
by vertex B) . Finding B, the region above AB and having segment AB 
as part of its perimeter is added to the current region. However, 
had E been the first vertex to the right, ERASER would have proceeded 
normally to vertex C. 

As ERASER operates it documents which heuristic it invokes for 
a decision. The best way to demonstrate and explain these Series One 
and Two heuristics. <xb listed above, is to examine the printout from 
ERASER for some scenes. 

The documentation ERASER yields is in three parts, 

1. After printing its own name all the dark regions handed to ERASER 
are listed with: 

a. a name, viz. SHADOWn, where n is a number; 

b, a list of the vertices bounding the region, given in counter- 
clockwise order. 

2, When each dark region is considered the work HEURISTIC (the main 
function) is printed followed by the name of the region being 
considered- Immediately after this will be three types of infor- 
mation: 
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a. (vl Tl v2 T2) — vl and v2 are the names of the vert ices 
being analyzed; Tl and T2 are the types of the previous 
vertex, vl, and the current vertex, v2, respectively. 

b. (Tl > T2) — Tl and T2 are the same vertex typos as in 

(a); however, this basic form will be in one of four formats. 
It shows whether a Primary or Secondary heuristic was applied 
(refer to Figure 12.1a). 

i. T1====>T2 j delete current line, primary (l)-(5) 
ii. <******ti - restore previous line, secondary (6) 
iii. ******>tj . delete previous line, secondary (7) 
iv. Tl******>T2 ; delete current line, secondary (8) 

c. {{CALL RGNl t) vl & v2) This is printed when a Series Two 
heuristic is used. If t is X, then {i) was usedr if t is T, 
(ii) was used; vl and v2 are two vertices in the perimeter 
of the region being added to the current shadow. (RGNl is 
the name of the function that actually does the merging.) 

3. The final part is a listing of the results and is selfexplanatory. 
Figure 13 shows the befo re and after plot of a simple scene. The 
ERASER printout is presented in Figure 14. For SHADOWl, the real 
body region, the no-action heuristics are (from the above charts): 
l(a);^2(b) (since the L is in the continuing line of the T) ; 5| 5j 5. 
shadow 2 is actually a shadow and three line segments are removed by 
these heuristics: l(a)j l(b)j 7; 2(b);^l(c)r 5. Examples of the 
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Series 1*wo heuristics can be found in the printout* for both Figures 
9 and 13. Figures 15 through 24 give the before/after plots of the 
five scenes 9hown earlier along with their diagnostic . The plots 
show the success of ERASER; SEE would now have no problems. 

All That Glitters... 

The debugging of ERASER led to a couple of thoughts about when 
it will lose* Very dark blocks are definitely bad news. A light 
block and a dark block side by side look locally like a light block 
with a shadow on it. For such a situation shown in Figure 25a, if 
ERASER stumbled onto the boundary between the blocks {segments AD 
and BC) , it would be removed- The aborted block of Figure 21 shows 
the shadow consideration of this problem. I presently see no method 
for locally deciding this problem. 

When a couple or three vertices are marked by the primary heuris- 
tics for removal and the continuing line rule remains satisfied, the 
line segments are quite dead. Various combinations of obscuring 
bodies will have the lethal sequence of vertex types if there are 
not sufficient shadows to break up the perimeters of the regions. In 
Figure 5, segment TU shows where a T-T sequence should be eliminated 
and segments AB and CD in Figure 25b show where the Bame sequence 
should not be removed. This again seems like a problem requiring a 
globa 1 solut ion * 
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How to Uaelt 

The complete ERASER is contained on the file named "ERASER n" 
where n is the highest integer found* Figure 26 is a printout of 
all the information about the scene of Figure 3 that is needed for 
successful operation of ERASER. Each vertex is named and has been 
given an x and y coordinate, a TYPE, a TYPE*, and a NEIGHBORS 
list. These properties are explained in the paper by Guzman. The 
argument to ERASER is a list of line segments. A segment is a line 
from the perimeter of one dark region, and is represented by the 
list of its two end points. ERASER expands the line segment into 
the complete perimeter of the dark region and begins ccmputation. 

If a vertex is the endpoint of a segment that is to be removed, 
the other endpoint of the segment is put on its property list under 

the indicator KILL. At segment removal time this KILL vertex is 

* 

removed from the NEIGHBORS list. 
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